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We claim: 

1 . A video processing device, comprising: 

a background audio cliange detecting means for detecting bacl<:ground audio 
changes in video data; and 

a memory communicating with said bac[<ground audio change detecting 
means and storing said video data and audio data corresponding to said video data; 

wherein said background audio change detecting means detects a 
background audio change in said audio data and detects semantically meaningful 
video scenes using detected background audio changes. 

2. The device of claim 1 , wherein said background audio change 
detecting means further delimits segments of said video data. 

3. The device of claim 1 , wherein said background audio change 
detecting means further determines if said audio data comprises background audio. 

4. The device of claim 1 , wherein said background audio change 
comprises an audio blank. 

5. The device of claim 1 , wherein said background audio change 
comprises an audio volume change. 

6. The device of claim 1 , wherein said background audio change 
comprises an audio frequency content change. 
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7. The device of claim 1 , wherein said background audio change 
comprises a start or stop of music. 

8. The device of claim 1 , wherein said video processing device comprises 
a video recorder device. 

9. The device of claim 1 , wherein said video processing device comprises 
a video editor device. 

1 0. The device of claim 1 , wherein said video processing device comprises 
a video authoring device. 
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11. A video processing device, comprising: 
a processor; 

a background audio change detector communicating witli said processor; and 
a memory communicating with said processor, said memory storing video 

data and audio data corresponding to said video data; 

wherein said background audio change detector detects a background audio 

change in said audio data and wherein said processor detects semantically 

meaningful video scenes using detected background audio changes and delimits 

segments of said video data. 



1 2. The device of claim 1 1 , wherein said memory stores a predetermined 
audio volume change threshold and wherein said background audio change detector 
comprises an audio volume detector, and wherein said audio volume detector 
generates an audio volume change value and detects a background audio change 
when said audio volume change value exceeds said predetermined audio volume 
change threshold. 



1 3. The device of claim 1 1 , wherein said memory stores a predetermined 
audio frequency change threshold and wherein said background audio change 
detector comprises a frequency change detector, and wherein said frequency 
change detector generates a frequency change value and detects a background 
audio change when said frequency change value exceeds said predetermined audio 
frequency change threshold. 
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14. The device of claim 1 1 , wherein said memory stores a predetermined 
audio blanl< volume threshold and a predetermined audio blank period threshold and 
wherein said background audio change detector comprises an audio blank detector, 
and wherein said audio blank detector generates an audio blank volume value and 
detects an audio blank when said audio blank volume value is less than said 
predetermined audio blank volume threshold for at least said predetermined audio 
blank period threshold. 

1 5. The device of claim 1 1 , wherein said memory stores a predetermined 
noise threshold and a predetermined music volume threshold and wherein said 
background audio change detector comprises a music detector, and wherein said 
music detector detects substantially stable frequency peaks in said audio data and 
detects music if a music volume is lower than said predetermined noise threshold 
and exceeds said predetermined music volume threshold. 

1 6. The device of claim 1 1 , wherein said video processing device 
comprises a video recorder device. 

17. The device of claim 1 1 , wherein said video processing device 
comprises a video editor device. 

1 8. The device of claim 1 1 , wherein said video processing device 
comprises a video authoring device. 
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1 9. The device of claim 1 1 , wherein said memory further stores a speech 
detection routine, and wherein said processor uses said speech detection routine to 
determine that said audio component is background audio if speech of a 
predetermined energy level Is not detected. 

20. The device of claim 1 1 , wherein said memory further stores a music 
detection routine, and wherein said processor uses said music detection routine to 
determine that said audio component is background audio if music of a 
predetermined signal-to-noise level is not detected. 
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21 . A method of segmenting video data, comprising the steps of: 
detecting background audio changes in bacl<ground audio data corresponding 

to said video data; 

detecting semantically meaningful video scenes using detected background 
audio changes. 

22. The method of claim 21 , further comprising the preliminary step of 
determining whether audio data of said video data is said background audio. 

23. The method of claim 21 , with the step of detecting background audio 

changes further comprising detecting an audio volume change in said background 
audio in excess of a predetermined audio volume change threshold. 

24. The method of claim 21 , with the step of detecting background audio 
changes further comprising detecting a frequency change in said background audio 
in excess of a predetermined audio frequency change threshold. 

25. The method of claim 21 , with the step of detecting background audio 
changes further comprising detecting an audio volume that is less than a 
predetermined audio blank volume threshold for at least a predetermined audio 
blank period threshold. 
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26. The method of claim 21 , with the step of detecting background audio 
changes further comprising the steps of: 

generating a frequency data from said audio data; 
detecting substantially stable frequency peaks in said frequency data; and 
detecting a background audio change at a start or stop of said substantially 
stable frequency peaks. 

27. The method of claim 21 , further comprising the step of delimiting video 
segments corresponding to said semantically meaningful video scenes. 

28. The method of claim 21 , further comprising the step of delimiting video 
segments corresponding to said semantically meaningful video scenes by inserting 
indexes into said video data at a beginning of said semantically meaningful video 
scenes. 

29. The method of claim 21 , further comprising the step of delimiting video 
segments corresponding to said semantically meaningful video scenes by extracting 
and storing one or more representative video frames from each video segment. 
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30. A method of segmenting video data, comprising tlie steps of: 
determining whetlier audio data corresponding to said video data is 

background audio; 

sampling said audio data If said audio data is said background audio; 
detecting bacl<ground audio changes in said audio data; 
detecting video shot changes in said video data; 

detecting semantically meaningful video scenes using detected background 
audio changes that occur in conjunction with detected video shot changes; and 

delimiting video segments corresponding to said semantically meaningful 
video scenes. 

31 . The method of claim 30, with the step of detecting background audio 
changes further comprising detecting an audio volume change in said background 
audio in excess of a predetermined audio volume change threshold. 

32. The method of claim 30, with the step of detecting background audio 
changes further comprising detecting a frequency change in said background audio 
in excess of a predetermined audio frequency change threshold. 

33. The method of claim 30, with the step of detecting background audio 
changes further comprising detecting an audio volume that is less than a 
predetermined audio blank volume threshold for at least a predetermined audio 
blank period threshold. 
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34. The method of claim 30, with the step of detecting baci<ground audio 
changes further comprising the steps of: 

generating a frequency data from said audio data; 
detecting substantially stable frequency peal<s in said frequency data; and 
detecting a background audio change at a start or stop of said substantially 
stable frequency peaks. 

35. The method of claim 30, with the step of delimiting video segments 
further comprising inserting indexes into said video data at a beginning of said 
semantlcally meaningful video scenes. 

36. The method of claim 30, with the step of delimiting video segments 
further comprising extracting and storing one or more representative video frames 
from each video segment. 
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